Zillow MMM - Visits
Spencer Smith
Generated on ‘21 January, 2020’
Model Hyper Parameters
Training dates 2016-09-01 to 2019-12-30
Transformations
| Channel | Diminishing Returns Shape | Decay rate |
|---|---|---|
| TV | log | 0.98 |
| Digital | 0.9 | 0.3 |
| SEM Brand | log | 0.4 |
| SEM Non-Brand | 0.9 | 0.4 |
| Social | 0.9 | 0.6 |
| App | 0.9 | 0.93 |
Model Performance
\(R^2\) = 0.9749618
MAPE = 2.47%
MAPE (Test) = 2.75%
RMSE = 462,689
RMSE (Test) = 492,347
| Estimate | t value | Pr(>|t|) | |
|---|---|---|---|
| (Intercept) | -5912360.44 * | -1.729 | 0.084 |
| tv | 947.433 * | 1.598 | 0.110 |
| semBR | 14704.205 * | 1.645 | 0.100 |
| semNB | 13.156 | 4.536 | 0.000 |
| app | 7.446 | 7.912 | 0.000 |
| social | 28.582 | 10.335 | 0.000 |
| digital | 15.682 | 3.586 | 0.000 |
| email_marketing | -0.019 * | -0.659 | 0.510 |
| email_operational | 0.145 | 7.002 | 0.000 |
| email_savedsearch | 0.146 | 10.572 | 0.000 |
| SEO_bigcity | 8308646.149 | 2.366 | 0.018 |
| SEO_hdp | -86569.851 * | -0.076 | 0.939 |
| push_android | -0.033 | -3.159 | 0.002 |
| push_ios | 0.125 | 8.878 | 0.000 |
| nonbrand_queries | 51.248 | 13.586 | 0.000 |
*Not significant effect
Lift Analysis
Lift is the expected contribution of the channel to the overall prediction. “If we removed the channel, what percent of total Visits would we lose?”| 2017 | 2018 | 2019 | |
|---|---|---|---|
| (Intercept) | -44% | -37.9% | -32.7% |
| tv | 4% | 3.1% | 2.8% |
| semBR | 2% | 1.4% | 1.2% |
| semNB | 2% | 2.3% | 1.9% |
| app | 4% | 4.7% | 6.3% |
| social | 6% | 6.2% | 6.9% |
| digital | 1% | 1.4% | 1.6% |
| email_marketing | 0% | -0.1% | -0.1% |
| email_operational | 5% | 2.6% | 3.3% |
| email_savedsearch | 4% | 5.6% | 4.9% |
| SEO_bigcity | 61% | 52.6% | 45.1% |
| SEO_hdp | -1% | -0.5% | -0.4% |
| push_android | -1% | -0.8% | -1.0% |
| push_ios | 2% | 2.9% | 4.7% |
| nonbrand_queries | 42% | 35.3% | 32.1% |
CPA Analysis
| date | tv | semBR | semNB | app | social | digital |
|---|---|---|---|---|---|---|
| 2016-09-01 | $0.16 | $0.04 | $0.13 | $0.02 | $0.04 | $0.11 |
| 2017-01-01 | $0.23 | $0.04 | $0.13 | $0.02 | $0.04 | $0.12 |
| 2018-01-01 | $0.26 | $0.04 | $0.13 | $0.03 | $0.04 | $0.12 |
| 2019-01-01 | $0.21 | $0.05 | $0.13 | $0.03 | $0.04 | $0.13 |
Error Plot
Variable Analysis
Variable Correlation
Media Marginal Plots
Variable plots
Owned
## [1] "email_marketing" "email_operational" "email_savedsearch"
## [4] "push_android" "push_ios"
Search
## [1] "SEO_bigcity" "SEO_hdp" "nonbrand_queries"
Apendix
Seasonal Variable
| Estimate | t value | Pr(>|t|) | |
|---|---|---|---|
| christmas | -2,222,067 | -11.797 | 0.000 |
| easter | -1,316,868 | -6.524 | 0.000 |
| fathers | -851,984 | -4.203 | 0.000 |
| halloween | -924,387 | -3.747 | 0.000 |
| independence | -2,235,368 | -7.768 | 0.000 |
| labor | -521,534 | -3.573 | 0.000 |
| martin | 314,085 * | 1.121 | 0.262 |
| memorial | -1,472,329 | -8.555 | 0.000 |
| mothers | -958,076 | -4.697 | 0.000 |
| new | -1,271,231 | -6.297 | 0.000 |
| presidents | 237,153 * | 0.844 | 0.399 |
| super | -829,013 | -2.938 | 0.003 |
| thanksgiving | -1,599,657 | -11.482 | 0.000 |
| valentines | -1,380,763 | -4.951 | 0.000 |
| year2017 | 884,350 | 10.702 | 0.000 |
| year2018 | 2,405,974 | 20.291 | 0.000 |
| year2019 | 3,353,969 | 19.862 | 0.000 |
| month2 | 1,090,472 | 13.320 | 0.000 |
| month3 | 1,500,339 | 18.681 | 0.000 |
| month4 | 1,956,723 | 21.389 | 0.000 |
| month5 | 1,536,695 | 16.354 | 0.000 |
| month6 | 1,152,536 | 11.416 | 0.000 |
| month7 | 1,681,297 | 16.724 | 0.000 |
| month8 | 1,598,737 | 17.838 | 0.000 |
| month9 | 917,856 | 11.517 | 0.000 |
| month10 | 788,983 | 9.028 | 0.000 |
| month11 | 212,252 | 2.085 | 0.037 |
| month12 | -802,582 | -8.478 | 0.000 |
| wdayMonday | 4,624 * | 0.058 | 0.953 |
| wdaySaturday | -216,518 | -3.090 | 0.002 |
| wdaySunday | -1,052 * | -0.012 | 0.991 |
| wdayThursday | 44,640 * | 0.654 | 0.513 |
| wdayTuesday | 44,313 * | 0.598 | 0.550 |
| wdayWednesday | 113,255 * | 1.604 | 0.109 |